![]()
Robert Axelrod
The One Shot Game
Axelrod worked with this version of Prisoners’ Dilemma (PD) (which you’ve seen before).
The trick was that each pair of people would play an iterated version of this game.
Indefinite Iteration
In the fancier version of the game, he didn’t tell people how long the game would go.
- Instead he just said there was a probability of it ending after each round; if I recall 0.005.
- This was used to avoid backwards induction reasoning, but it was unnecessary.
Backwards Induction
Some of you might have come across this problem in the 5 round game.
- A big benefit of cooperating is triggering future cooperation.
- But in round 5 there’s no future, so why cooperate then?
- If everyone knows that reasoning works, why cooperate in round 4? Etc back to never cooperating.
- This reasoning doesn’t work in practice, and it’s a fascinating question whether it works in theory.
The Tournament
- There are n strategies submitted.
- The strategies are computer programs, which say what to do at each stage given the history of the game.
- Each will play k rounds of PD with each of the other n-1 strategies.
- Their payouts will add up over the \(k(n-1)\) rounds and the one with the highest total will win.
Cooperative and Competitive
- This is not entirely a cooperative game; ultimately if I’m a strategy I want to win, and that means I want the other strategy I’m interaction with to lose.
- But in the short run there is much to be gained by improving our mutual position vs the other \(n-2\) strategies.
- So in the short run there is a benefit to cooperation, even if we’re ultimately rivals.
The First Tournament
- Axelrod advertised the first round of his tournament, and called for submissions.
- This was far from trivial in pre-internet days, and he only got 13 submissions.
- In the first tournament he said that \(k\) would be 100, but no one actually exploited that fact.
Question
What kind of strategy would you endorse?
Would the programming language make a difference?
Tit-for-Tat
Two rules.
- Play C at round 1.
- In all subsequent rounds, do whatever the other player just did.
The Second Tournament
- So Axelrod wrote this up, including saying who won.
- He called for more submissions, and now got 66.
- Some of these were typed, some came to Ann Arbor on the huge magnetic disks that were used way back then.
- He ran the tournament again, this time with a random number of rounds.
- And Tit-for-Tat won again.
Logic and Victory
- This doesn’t mean Tit-for-Tat is the best strategy.
- Indeed, in each tournament it was easy in retrospect to describe strategies that would have beaten everyone, including TFT, if they had been entered.
- But still, it’s pretty impressive.
Four Features
Tit-for-Tat has five striking characteristics, each of which was positively correlated with success in the tournaments.
- Nice
- Provocable
- Forgiving
- Not envious
- Simple
Nice
The clearest distinction in the tournament was between strategies that were Nice and those that were Nasty.
- By definition, a strategy is Nice iff it is never the first to defect.
- You don’t have to be very nice in the intuitive sense to count as Nice.
Grim Trigger
Here is one nice strategy, one Axelrod calls Grim Trigger.
- Cooperate on move 1.
- If the other player ever defects, defect on every subsequent move.
This strategy did really badly; it was the worst Nice strategy in round 2. But still many Nasty strategies did worse.
Nice Strategies
- In the evolutionary versions of the game, there can be a tendency for strategies to tend towards being Nice.
- Then evolution stops, because when two Nice strategies meet, the payout is inevitably 3k to each.
- Although the best strategies are all Nice, it is how they interact with Nasty strategies that determines who wins.
Provocable
- It’s bad to get pushed around.
- Nasty strategies are always looking for how much they can get away with.
- So you want to send a clear message that defections will not be tolerated.
- Obviously TFT does that.
Forgiving
- But you don’t want to be Grim Trigger.
- It’s bad to be pushed around, but it’s not much better to end up in all defect land.
- You need a way back to all cooperate land.
- TFT has that, though notably it isn’t perfect at this.
- TFT can get into CD-DC-CD-etc cycles with a bunch of strategies.
Not Envious
- In any interaction, TFT never does better than who it is playing with.
- Yet it comes out first overall.
- This is kind of amazing.
- It just does not care at all about winning against who it is facing off with.
Not Envious To a Fault
- Note that TFT doesn’t always do that well in evolutionary games. (If we get time we’ll come back to what I mean by this.)
- This is because it might take this a bit too far.
- It doesn’t look to exploit weaknesses in opponents.
Simple
- Other strategies try to figure out what their rivals are doing.
- They normally get this wrong.
- Or they try and send complex signals.
- These are usually misinterpreted.
- TFT keeps things simple, and doesn’t lose points messing around looking for any edges.
Variant Games
- The most interesting variant to me is the one where a strategy only gets implemented with probability 0.99 on each move.
- Sometimes there are performance errors.
- TFT does terribly in this; it can’t get out of randomly generated defection cycles.
Variant Games
- In this kind of game you need to be a bit more forgiving.
- But also you can try to get away with a bit more; if the other person will treat a defection as random, you can plan a few.
What Difference Does it Make
- Same game as before - a 5-round Prisoners’ Dilemma
- Go to https://veconlab.econ.virginia.edu
- Login as participant
- Session name: pbw2
- If this doesn’t work, we’ll go back to other things.
Three Kinds of Game Theory
- Rational Choice
- Evolutionary
- Experimental
Rational Choice
Assume common knowledge of rationality, see what you can figure out about how the game will go.
This is primarily what Bonanno’s book is about.
There is no standard name for this, it’s often just called ‘game theory’. Like with Apple product naming, it’s very unhelpful to have one variant have a null modifier, so I use expressions like ‘rational choice game theory’.
Evolutionary
Assume strategies are hard-wired and interactions are random.
Also assume that populations of each kind (i.e., each strategy) in generation (n+1) are proportional to
- Population of that kind in generation n;
- Average score of that kind in generation n.
In this game, you might have 100 generations where each generation is 100 rounds of PD.
Experimental
Put people (and as always by people here we usually mean undergrads at fancy universities) in labs and see what happens.
PD is interesting to all three kinds.
For Next Time
Next week we’ll go over two central notions of rational choice game theory:
- Iterated deletion
- Backwards induction
Social Challenge